Volumetric image enhancement system and method

ABSTRACT

A technique is described for enhancing three dimensional or volumetric image data, such as thick slice or slab data, or data for slices stacked in a third dimension orthogonal to an imaging plane. The technique processes the image data by reference to data parameters both in two dimensions and in three dimensions. The processing permits identification of structural pixels and their differentiation from non-structural pixels. The structural pixels may be identified by reference to gradients determined in three dimensions, but with directions determined by reference to only two dimensions. The structural and non-structural pixels may then be processed differently to provide an enhanced image.

BACKGROUND

The present invention relates generally to the field of digital imagingand particularly to enhancement of volumetric images. More particularly,the invention relates to techniques for processing images in acombination of two-dimensional and three-dimensional analysis steps.

Digital images are typically made and processed in either two or threedimensions. Two-dimensional (2D) images are conventionally made byimpacting a digital detector with radiation. The detector may be acharge coupled device, as in digital cameras, or a more complex detectorsuch as those used in digital X-ray and various radiographic techniques.Three-dimensional (3D) image data may be acquired as a plurality of 2Ddatasets. Techniques for acquiring 3D images include magnetic resonanceimaging (MRI), computer tomography (CT) imaging systems, and so forth.

Techniques have also been developed for enhancing 2D and 3D images. Ingeneral, these techniques are specifically adapted to either 2D imageenhancement or 3D image enhancement. Assumptions made in eithertechnique may generally hold valid for specific situations only,however. For example, for images having pixels of a first pitch orspecial resolution (i.e., the number of pixels per unit length or area),in both an image plane and in a third dimension, existing techniques mayperform adequately. However, where the dimensions are different,information may be lost or analysis of the content of the images may bedistorted. This is particularly true of images having a greater depth ina direction orthogonal to an image plane. Such images may be termed“thick slice” or “thick volumetric” images.

Several 2D and 3D image enhancement frameworks have been proposed forenhancing 2D and 3D images. In general, such enhancement techniques areuseful for identifying features and objects of interest, typicallyvisible objects in the images. Depending upon the context, such featuresmay be circumscribed, identified, categorized, and analyzed, such as forrecognition purposes. In a medical diagnostic context, for example,various anatomies and disease states may be determined based upon suchimage analysis. The analysis may similarly be use for visualization ofstructures and anatomies. In other contexts, such as part inspection,defects and internal features may be visualized and analyzed in asimilar manner. Still further, in contexts such as baggage and parcelinspection, the internal contents of objects may be determined byanalysis and recognition techniques.

Thick volumetric images have characteristics of both 2D and 3D images.For such images, 2D filtering does not make full use of a thirddimension. That is, sampling in a third dimension, which may be referredto as a “Z direction”, may be rather poor in such image data, resultingin relatively poor results of the analysis when using 3D filtering.

There is a need, therefore, for improved techniques for analyzing thickvolumetric images. There is a particular need for a technique that canmake use of additional information provided by volumetric data whilestill maintaining full use of in-plane data. Such techniques couldenhance details by highlighting areas of visual interest better than 2Dtechniques.

BRIEF DESCRIPTION

The present invention provides novel image enhancement techniquesdesigned to response to such needs. The techniques are particularlywell-suited to enhancement of thick volumetric images, that is, imageshaving a sampling along a dimension orthogonal to an imaging plane thatis greater than, and particularly over five times that along in-planedimension. However, the invention is not necessarily limited to anyparticular thickness, but may be more generally suited to analysis ofvolumetric images including image data available in two and threedimensions. The technique allows for analysis of fine structures thatare sparsely sampled and generally not analyzed or insufficientlyanalyzed in 3D processing alone. Larger structures are still adequatelysampled, however, and can be used for 3D processing in the presenttechnique. The present image filtering framework thus performs analysisvia steps in both 2D and 3D processing. The technique may be referredto, therefore, as “2.5D”.

The technique may be used in a number of contexts, such as conventionalmedical imaging, as well as for part inspection and analysis, baggageand parcel inspection and analysis, and so forth. Moreover, thetechnique is not limited to any particular imaging modality, but may beused with a range of existing modalities. Finally, the technique may beemployed directly at an image data acquisition system, or may be used inconjunction with a processing station that may be entirely remote fromthe image data acquisition system, which may process image datasubsequent to image data acquisition, from image data stored in amemory.

In accordance with certain aspects of the techniques, processing stepsare performed in reference to 3D image data, while other processingsteps are performed in reference to 2D image data. In particular,analysis and segmentation of the structural features in the image dataare performed by a combination of 2D and 3D processing. The processingrenders a map or binary mask of pixels that are determined to berepresentative of structure and pixels that are determined to berepresented as of non-structure. The image data may then be furtherprocessed based upon this categorization.

DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a diagrammatical representation of an exemplary imaging systemfor acquisition of 2D and 3D image data which is processed in accordancewith the present technique;

FIG. 2 is an exemplary 3D imaging volume used to make a series of imagesenhanced by the present technique;

FIG. 3 is a diagrammatical representation of exemplary logic forcarrying out the 2.5D image enhancement scheme of the present technique;

FIG. 4 is a perspective view of an exemplary portion of an imagingvolume used to illustrate how a block or neighborhood of pixel data isemployed in the analysis of FIG. 3;

FIG. 5 is a similar perspective view with certain neighboring pixelsremoved to permit viewing of an internal pixel of interest;

FIG. 6 is a diagrammatical representation of exemplary steps in logicfor the analysis and segmentation summarized in FIG. 3;

FIG. 7 is a depiction of a series of matrix operators employed for theanalysis of FIG. 6; and

FIG. 8 is perspective view of a reduced neighborhood set for certain ofthe analysis performed in accordance with the steps of FIG. 6.

DETAILED DESCRIPTION

Turning now to the drawings and referring first to FIG. 1, an imagingsystem 10 is illustrated generally as including an imager 12 forcreating image data of a subject 14. Although a human figure isgenerally shown as the subject 14, it should be borne in mind that anyappropriate subject could be imaged. In the present context, forexample, the subject may be human or animal, animate or in-animate, suchas manufactured parts, naturally occurring subjects and so forth.Indeed, the imaging system 10 may be any suitable type of system thatproduces digitized image data based upon some imaging physics. In themedical imaging context, as elsewhere, such imaging systems may includeMRI systems, PET systems, CT system, tomosynthesis systems, X-raysystems, ultrasound systems, among many other imaging modalities. Thesystems may also include conventional photographic imaging systems thatproduce digitized image data based upon received radiation of anysuitable bandwidth or frequency.

In the diagrammatical view of FIG. 1, the imaging system includes animager 12 coupled to imager control circuitry 16 and image dataacquisition circuitry 18. Depending upon the modality and physics of thesystem, the imager will typically either emit some type of radiation, aswith X-ray, CT, tomosynthesis, and other systems. Other active imagingsystems, such as MRI systems, influence subjects by excitation, such asthrough generation of radio frequency pulses in the presence ofcontrolled magnetic fields. In all these cases, however, the imager isregulated in its operation by the imager control circuitry 16. Suchcontrol circuitry may take any suitable form, and typically includescircuitry for activating the imager, receiving radiation or othersignals, creating any excitation signals or radiation required forimaging, and so forth. The image acquisition circuitry 18, then,receives and initially processes data received by imager. Such initialprocessing may include conversion of analog signals to digital signals,filtering of the analog or digital signals, scaling or dynamic rangeadjustments, and the like.

The image control circuitry 16 and the image data acquisition circuitry18 are generally regulated by some type of system control circuitry 20.Again, depending upon the nature of the imaging system and the physicsinvolved, the system control circuitry may initiate imaging sequences byexchanging appropriate signals with the imager control circuitry 16. Thesystem control circuitry 20 may also receive the raw or pre-processedimage data from the image data acquisition circuitry 18. The systemcontrol circuitry 20 may, particularly in more complex systems, becoupled to an operator workstation 22 where an operator selects,configures, and launches examination or imaging sequences. The imagedata, either raw, partially processed or fully processed, is typicallystored in some type of storage media as represented at reference numeral24. In the present context, such storage media may be part of the systemcontrol circuitry 20, the operator workstation 22, or any othercomponent of the overall system. In a medical diagnostics context, forexample, such storage media may include local and remote memory, bothmagnetic and optical, and may include complex picture archive andcommunication systems (PACS) designed to store and serve image data upondemand.

In the illustration of FIG. 1, the operation workstation 22 is shown ascoupled to image data processing circuitry 26. Again, such processingcircuitry may actually be distributed throughout the system, and mayembody hardware, firmware, and software designed to process the imagedata to produce reconstructed images for viewing. In accordance with thepresent techniques described below, the image processing circuitry 26performs a combination of 2D and 3D (“2.5D”) data processing on theimage data to manipulate the raw or pre-processed data so as to furtherenhance the usability of the reconstructed images. The image dataprocessing circuitry 26 may be local to the imaging system, asillustrated generally in FIG. 1, or may be completely remote from thesystem, and simply access the image data, as from the storage media 24for post-processing. Finally, FIG. 1 illustrates various remotecontrol/processing/viewing stations 28 that can be coupled to theimaging system by appropriate network links 30. Such stations may beused for further viewing, analyzing, and processing the image data asdescribed herein.

As will be appreciated by those skilled in the art, image data of thetype acquired on many different imaging systems is generally processedby various filtering, and enhancement techniques to recognize, clarify,or otherwise manipulate discrete picture elements or pixels that areencoded by the image data. Each pixel typically includes datadescriptive of its location and its intensity, or in color systems, itsintensity in several different colors. The enhancement of the image datais performed by mathematically computing various characteristics of eachpixel and of neighboring pixels so as to recognize and treat the pixelsin useful groups. Essentially, the image data is most useful when ahuman or machine can recognize groups of pixels that share certainqualities or characteristics. For example, structures and non-structuresmay be recognized in image data by the processes described below. Thestructures may represent edges, surfaces, contours, regions, colors, andother image features that, when viewed by a human or machine viewer,render the image useful or subject to some type of interpretation.

FIG. 2 represents an exemplary imaging volume, such as imaging system 10described above. As recognized by those skilled in the art, the imagingvolume may be analyzed in three orthogonal directions, including anx-direction 32, a y-direction 34 and a z-direction 36. The present 2.5Dimage enhancement technique or framework enhances thick volumetricimages made in the imaging volume by essentially treating such images as3D objects for analyzing and segmenting purposes to obtain a 3Dsegmentation mask. However, the framework treats each of the thickslices or images separately for enhancement filtering in 2D while guidedby the 3D segmentation mask. As a result, the technique makes use of 3Dinformation for performing 2D filtering. Therefore, the technique hasthe potential to further improve the robustness of image filtering.

In the illustration of FIG. 2, a slab or slice 38 of the imaging volumeis represented as including discrete volume elements (voxels) which are,for the present purposes, referred to as picture elements (pixels) 40 inthe selected slap or slice 38. The pixels have a known pitch ordimension in each orthogonal direction, including a pitch 42 in thex-direction, a pitch 44 in y-direction, and a pitch 46 in thez-direction. As will be appreciated by those skilled in the art, a 2Dimage generally represents the projection of three dimensions into a 2Dimaging plane. An image 48 may thus be presented as the projection ofthe pixels 40 of the slab or slice 38 in an imaging plane at the surfaceof the slab or slice. As will be appreciated by those skilled in theart, however, where the dimension or pitch 46 in the z-direction differsfrom that in the x- and y-directions, the projection may not fullyrender certain of the details that represent features or structuresextending back in the z-direction. The image data is processed in thepresent context in such a manner as to more accurately analyze andprocess structural features taking such detail into account.

Exemplary logic for carrying out certain of the image enhancement stepsaccording to the present technique is illustrated in FIG. 3 anddesignated generally by the reference numeral 50. The processing logicbegins with the input or access to 3D image data as indicated at step52. As will be appreciated by those skilled in the art, the image datatypically includes a series of values, digitized over a useful dynamicrange. Each pixel in the image data corresponds to one of the image datavalues. The image data itself may be included in a larger image datastream, which may include such other features as descriptive headers,identification data, date tags, compression routine identifiers, and soforth. In the present context, the image data includes a pixel datawhich may be referred to as I₁(x,y,z), where the term “I₁” representsthe collective values of each pixel in the image at individual locationsdefined by the parenthetical.

At step 54, boundaries of the image are extended by mirroring. As willbe appreciated by those skilled in the art, such mirroring permitsanalysis and computations to be made on pixel data near boundaries ofthe image, particularly where spatial filters are employed as describedbelow. In the present context, the mirroring is performed by addingpixels to each side of the 3D image with a first added pixel adjacent toeach image pixel being equal in value to the image pixel itself, and asecond adjacent at a pixel being equal to the pixel value of theimmediately adjacent pixel in the image data. Such mirroring may be donefor any suitable number of rows or columns of the image data. In apresent implementation, a kernel having dimensions of 3×3 is employed asdescribed below, such that mirroring is performed by adding two rows orcolumns or slices in each of the three image dimensions.

At step 56, the image is shrunk to facilitate analysis. The shrinking isperformed by replacing each pixel with the average value of neighboringpixels. The number of pixels considered for averaging may depend upon ashrink factor which may be programmed into the present algorithm, or maybe selected by a user. In a present implementation, shrinking isperformed two dimensionally for each slice of the image data. Thosestill in art will also recognize that the mirroring performed at step 54facilitates computation of the pixel average values for those pixelsnear the image boundaries. The resulting shrunk image data is referredto as I₂(x,y,z).

At step 58 in FIG. 3, analysis and segmentation are performed on theshrunk image I₂(x,y,z). The analysis and segmentation are intended togenerate a mask or map of structural pixels and non-structural pixels.That is, pixels considered to be representative of features of interestin the image, typically edges and more prominently contrasting details,may be termed “structural.” While other features, including texture, maynot be considered structural, they may nevertheless be important foroverall image analysis, visualization, and so forth. However, processingof structural pixels differently from non-structural pixels has beenfound to be particularly useful in developing enhanced images. Beforeproceeding to the other steps summarized in FIG. 3, the detailed stepsinvolved in the analysis and segmentation performed at step 58 aredescribed with particular reference to FIG. 6.

The analysis summarized in FIG. 6 makes use of pixel neighborhoods thatmay be visualized as illustrated in FIGS. 4 and 5. In particular, thepixels of an individual image or slice of the imaging volume maygenerally fall within one plane, such as the middle vertical plane ofpixels shown in FIGS. 4 and 5. The volume portion 74, then, consists ofthis plane of pixels and additional pixels on either side of the plane.FIG. 5 illustrates the volume portion 74 with certain pixels removed toshow a center pixel 76 which, in the analysis described below, may beany pixel in the image, and is referred to as the pixel of interest. Thepixel of interest 76, then, may include as many as 26 neighboringpixels. That is, pixels 78 neighbor the center pixel 76 in anx-direction, pixels 80 neighbor the center pixel 76 in a y-direction,and pixels 82 neighbor the center pixel 76 in a z-direction. Throughoutthe present discussion, reference will be made to this neighborhood as a3×3 neighborhood, although larger neighborhoods may be employed for theprocessing.

Referring now to FIG. 6, exemplary steps in the analysis andsegmentation illustrated in FIG. 3 are presented. In the illustratedanalysis, the shrunk image I₂(x,y,z) is first scaled. This scaling isperformed by identifying maximum and minimum values for all pixels inthe image, referred to in a present implementation as “maxorig” and“minorig,” respectively. In a present implementation, a scale factor isdetermined by dividing a dynamic range value by the value of maxorig,with a present dynamic range value equally 4095. Other values may, ofcourse, be employed. The scaled value of each pixel, then, is computedby subtracting the value of minorig from each individual pixel value,and multiplying the resulting difference by the scale factor.

The scaled image is then smoothed as indicated at step 86 in FIG. 6. Ina present implementation, this smoothing is performed via a 3D boxcarfilter wherein each pixel becomes the center pixel or pixel of interestas illustrated in FIGS. 4 and 5 above, and each pixel value issequentially replaced by the average value of all pixels in the 3×3×3neighborhood. The smoothing is performed based upon the values of pixelsin the image I₂(x,y,z).

At step 88, the output of the boxcar filter is used to calculategradients for each pixel. As will be appreciated by those skilled in theart, in the digital imaging context, a “gradient” represents adifference between the value of a pixel of interest and a value of oneor more additional (i.e., neighboring) pixels. In step 88, multiplegradients are calculated for components in x, y and z directions. Thecomponents in the X, Y and Z directions called in a presentimplementation “gradX,” “gradY” and “gradZ” are calculated using theSobel 3D gradient operators set forth in FIG. 7. As those skilled in theart will recognize, the Sobel operators 102 are defined by a series ofmatrices 104 which may be multiplied by pixels at various locations inthe 3×3×3 neighborhood to obtain the component values. Based upon thesecomponent values, then, a gradient magnitude for the pixel of interestmay be calculated in 3D in accordance with the relationship:gradient magnitude=sqrt(gradX ²+gradY ²+gradZ ²)

While the gradient magnitude is computed in reference to the 3Dcomponents, however, its direction is computed in reference to only twodimensions, those defining the image plane or projection. In the presentcontext, the direction is computed in accordance with the relationship:direction=arctan (gradY/gradX).

Once the gradient magnitudes and directions have been computed for eachpixel in the shrunken image, the maximum and minimum gradients aredetermined, and a gradient histogram is obtained as indicated at step 90in FIG. 6. The maximum minimum gradients are determined by reference tothe gradient magnitudes. As will be appreciated by those skilled in theart, the gradient histogram, then, defines the number or counts ofindividual pixels having particular gradients magnitudes. In a presentimplementation, the histogram includes gradients for all voxels, eachvoxel having one gradient magnitude associated with it. Maximums andminimums are computed over each gradient value of all voxels. Thehistogram may be represented as a bar graph with gradient magnitudesalong a horizontal axis and counts or number of pixels having eachindividual gradient magnitude along a vertical axis. In practice,however, the processor simply stores addresses of pixels, their gradientvalues, and the counts for each magnitude.

At step 92, an initial threshold value is chosen and candidatestructural pixels are selected. The threshold value employed at thisstep is an initial gradient threshold value which is chosen initially ata value such that the gradient magnitude of 30% of the pixels in theimage I₂(x,y,z) will lie below the threshold. All pixels having valuesthat lie above the initial gradient threshold are then counted. In apresent implementation, a directional criteria is also imposed. That is,for a pixel to be counted as a candidate structural pixel, the gradientmagnitude of the pixel must lie above the initial gradient threshold,and the gradient direction of the pixel must be above a particularangle, termed “gradangle” in a present implementation. If the count isabove a predetermined value or count number, the pixel is retained as apossible relevant edge. The collection of relevant edge pixels is thennoted, such as in a mask or map wherein candidate structural pixels havea value of “1” and candidate non-structural pixels have a value of “0”.

At step 94 in FIG. 6, certain pixels of the mask are eliminated basedupon connectivity. In this process, each pixel having a value of 1 inthe binary mask is assigned an index number beginning with theupper-left hand corner of the image and proceeding to the lower right.The index numbers are incremented for each pixel having a value of 1 inthe mask. The mask is then analyzed row-by-row beginning in the upperleft by comparing the index values of pixels within small neighborhoods.For example, when a pixel is identified having an index number, afour-connected comparison is carried out, wherein the index number ofthe pixel of interest is compared to index numbers, if any, for pixelsimmediately above, below, to the left, and to the right of the pixel ofinterest. The index numbers for each of the connected pixels are thenchanged to the lowest index number in the connected neighborhood. Thesearch, comparison and reassignment then continues through the entirepixel matrix, resulting in regions of neighboring pixels being assignedcommon index numbers. In the preferred embodiment the index numbermerging may be executed several times. Each subsequent iteration ispreferably performed in an opposite direction (i.e., from top-to-bottom,and from bottom-to-top).

Following the iterations accomplished through subsequent search andmerger of index numbers, the index number pixel matrix will containcontiguous regions of pixels having common index numbers. A histogram isthen generated from this index matrix by counting the number of pixelshaving each index number appearing in the index matrix. As will beapparent to those skilled in the art, each separate contiguous region ofpixels having index numbers will have a unique index number. Regionsrepresented by index numbers having populations lower than a desiredthreshold are eliminated from the definition of structure. In apresently preferred embodiment, regions having a pixel count lower than50 pixels are eliminated. The number of pixels to be eliminated in thisstep, however, may be selected as a function of the matrix size, and theamount and size of isolated artifacts to be permitted in the definitionof structure in the final image.

At step 96, a final threshold is set for structural pixels and thestructure mask is refined based upon this threshold. Again, thestructure mask or map will generally correspond to a listing of thepixels in the image I₂(x,y,z) that are considered representative ofstructures in the image. Again, each pixel may be represented in thestructure mask as a “1” or a “0” such that the structure mask, whereinthe value “1” represents that the individual pixel is determined torepresent structure. The final gradient threshold set at step 96 fordetermination of the structure mask includes the addition of a desirednumber of pixels to the number labeled as “1” in the mask (the binarymask count). For example, in a presently preferred embodiment a value of4,000 is added to the binary mask count to arrive at a desired number ofpixels in the image structure definition. This parameter may be set as adefault value, or may be modified by an operator. In general, a higheradditive value produces a sharper image, while a lower additive valueproduces a smoother image. This parameter, referred to in the presentembodiment as the “focus parameter” may thus be varied to redefine theclassification of pixels into structures and non-structures.

The final gradient threshold or FGT is then determined. In particular,the population counts for each gradient magnitude value beginning withthe largest magnitudes are summed moving toward the lowest magnitudes.Once the desired number of structural pixels is reached (i.e., thenumber of pixels counted plus the focus parameter), the correspondinggradient magnitude value is identified as the final gradient threshold.In the presently preferred embodiment, the FGT value is then scaled bymultiplication by a value which may be automatically determined or whichmay be set by a user. For example, a value of 1.9 may be employed forscaling the FGT, depending upon the image characteristics, the type andfeatures of the structure viewable in the image, and so forth. The useof a scalable threshold value also enables the technique to be adaptedeasily and quickly to various types of images, such as for MRI datagenerated in systems with different field strengths, CT data, and soforth.

At step 98, pixels are eliminated based upon connectivity. In thepresent implementation, pixels are eliminated at this step if theindividual pixel has less than a defined threshold count from the maskusing a 6-connectivity approach. That is, each pixel is considered acentral pixel in a reduced neighborhood, with only pixels bordering thepixel of interest at a face are considered. FIG. 8 represents such areduced neighborhood in which a pixel of interest or central pixel 76 isshown and the reduced neighborhood of pixels at its faces are designatedby reference numeral 106. Considering such a reduced neighborhood,pixels are eliminated by a process otherwise identical to that describedabove with reference to step 94.

At step 100, small or isolated regions or volumes of pixels may berecuperated or included in the collection of pixels determined to bepotential structures, to provide continuity of edges and structures. Inthis process, a threshold is determined that may be referred to as afollower ratio, as a percentage of the final gradient threshold. If apixel is above the follower ratio, and is connected to the pixels whichare above the final gradient threshold (i.e., candidate structuralpixels), then the corresponding binary value of the pixel is changed.Such gradient following is generally carried out recursively in allthree dimensions, resulting in an initial classification of the pixelsas structural or non-structural.

Based upon such determinations, processing continues as indicated atstep 60 in FIG. 3. That is, once the analysis and segmentation of theindividual pixels are made based upon the 2.5D approach, a structuremask is obtained at step 60. This structure mask is obtained byrepeating the process of obtaining a final binary mask that describesstructural and non-structural regions of each projection image in theimaging volume. The final binary mask is obtained by comparing the pixelcount with a preset threshold, termed in a present implementation as“countthreshold1”. That is, if a current pixel has a binary mask valueof 1, the neighborhood pixel count is computed. If the pixel count isabove the threshold, the current pixel value in the mask is set to 1.Otherwise it is set to 0. If the current pixel value is 0 in the mask,the same account is computed and if it is above a lower threshold,termed in a present implementation “countthreshold2”, the binary valuefor the pixel in the mask is set to 1. Otherwise it is set to 0. Theresulting map of binary values represents the determinations of thepixels representing structure (those having a value of 1), and thoserepresenting non-structure (those having a value of 0).

At step 62 in FIG. 3, the shrunk image I₂(x,y,z) is filtered. In apresent implementation, three types of filtering are performed on theimage data, including orientation smoothing, homogenization smoothingand orientation sharpening. It should be noted that this filtering isperformed based upon the structure mask determined at step 60. That is,although the filtering is performed in two dimensions, whether thefiltering is performed and how the filtering is performed is based uponthe structure as determined by reference to two and three dimensionalanalysis.

In the orientation smoothing that may be performed at step 62, the valueof certain pixels is replaced with a value that is the average of itselfand those of neighboring pixels. In particular, depending upon whetherdominant orientation smoothing or local orientation smoothing isdesired, processing is as follows. If dominant orientation smoothing isselected, smoothing begins with assigning directional indices to eachpixel identified as a structural pixel. In the present embodiment, oneof four directional indices is assigned to each structural pixel inaccordance with the statistical variances for each pixel. Within a localneighborhood surrounding each structural pixel, statistical variancesfor pixel kernels in four directions are computed by reference to thenormalized intensity values of the surrounding pixels. The direction ofthe minimum variance is selected from the four computed values and acorresponding directional index is assigned. In the present embodimentthese directional indices are assigned as follows: “1” for 45 degrees;“2” for 135 degrees; “3” for 90 degrees; and “4” for 0 degrees. A localarea threshold value is assigned based upon the image matrix size. Inthe present embodiment, a local area threshold of 6 is used for 256×256pixel images, a value of 14.25 is used for 512×512 pixel images, and avalue of 23 is used for 1024×1024 pixel images. A dominant orientationis established for each structural pixel by examining the directionalindices within a local neighborhood surrounding each structural pixel.In this process, the directional indices found in the local neighborhoodare counted and the pixel of interest is assigned the directional indexobtaining the greatest count (or the lowest index located in the case ofequal counts).

In the present embodiment, both the dominant direction and itsorthogonal direction are considered to make a consistency decision inthe dominant orientation smoothing operation. These directions are 1 and2, or 3 and 4. It has been found that considering such factorssubstantially improves the robustness of the dominant orientationdetermination in the sense of being consistent with the human visualsystem (i.e. providing reconstructed images which are intuitivelysatisfactory for the viewer).

The consistency decision this made may be based upon a number ofcriteria. In the present embodiment, the image is smoothed along thedominant direction (i.e., the direction obtaining the greatest number ofcounts in the neighborhood) if any one of the following criteria is met:(1) the number of counts of the orientation obtaining the greatestnumber is greater than a percentage (e.g., 67%) of the totalneighborhood counts, and the orthogonal orientation obtains the leastcounts; (2) the number of counts of the orientation obtaining themaximum counts is greater than a smaller percentage than in criterion(1) (e.g., 44%) of the total neighborhood counts, and the orthogonaldirection obtains the minimum number, and the ratio of the counts of thedominant direction and its orthogonal is greater than a specified scalar(e.g., 5); or (3) the ratio of the dominant direction counts to itsorthogonal direction counts is greater than a desired scalar multiple(e.g., 10).

In the present embodiment, the neighborhood size used to identify thedirection of dominant orientation is different for the series of imagematrix dimensions considered. In particular, a 3×3 neighborhood is usedfor 256×256 pixel images, a 5×5 neighborhood is used for 512×512 pixelimages, and a 9×9 neighborhood is used for 1024×1024 pixel images.

Subsequently, the count determined in the searched neighborhood for eachpixel is compared to the local area threshold. If the count is found toexceed the local area threshold, the intensity value for each structuralpixel is set equal to the average intensity of a 1×3 kernel of pixels inthe dominant direction for the pixel of interest. Subsequently, thevalue of a corresponding location in the binary matrix structure mapchanged from 0 to 1. If the count is found not to exceed the local areathreshold for a particular pixel, the intensity value for the pixel ofinterest is set equal to a weighted average determined by therelationship:weighted avg=(1/(1+p))(input)+(p/(1+p))(smoothed value);where the input value is the value for the pixel of interest at thebeginning of smoothing, p is a weighting factor between 1 and 200, andthe smoothed value is the average intensity of a 1×3 kernel in thedominant direction of the pixel of interest.

Subsequently, the values of each pixel in the binary structure mask areevaluated. If the value is found to equal zero, the correspondingintensity value is multiplied by a weighting factor α. In the presentembodiment, factor α is set equal to 0.45. The resulting value is summedwith the product of the normalized intensity value for the correspondingpixel and a weighting factor β. In the present embodiment, the factors αand ⊕ have a sum equal to unity, resulting in a value of β equal to0.55.

If the value for a particular pixel is found to equal 1 in the binarystructure mask, the processor determines whether the desired number ofiterations have been completed, and if not, returns to further smooththe structural regions. In the present embodiment, the operator mayselect from 1 to 10 such iterations.

Homogenization smoothing is performed on non-structure pixels. In oneimplementation, the homogenization smoothing iteratively averages eachnon-structural pixel based upon a 3×3 kernel.

Orientation sharpening is performed on the orientation filtered orsmoothed structural pixels. Orientation sharpening is performed forthose pixels having gradients above a pre-specified limit, such as twicea final gradient threshold used for determining the structural pixels.

Following the filtering operations, processing may include renormalizingthe pixels. Such renormalization is performed by computing the averagepixel intensity in the filtered image I₁(x,y,z) and obtaining anormalizing factor based thereon. In accordance with a presentimplementation, the normalizing factor may be computed for each pixel bydividing the mean value before filtering by the mean value afterfiltering. The normalized image, then, is obtained by multiplying eachpixel value in the original image I₁(x,y,z) by the normalization factor,and then adding the minimum value minorig.

The filtered and renormalized image values, then, are expanded to bringthe filtered image back to its original size. This step, summarized atreference numeral 64 in FIG. 3 essentially reverses the shrinkingperformed at step 56. As will be appreciated by those skilled in theart, such expansion may be performed by bi-cubic or bilinearinterpolation. The image expansion is performed two dimensionally.

At step 66 texture blending is performed to recapture certain of thetexture present in the original data. In the present implementation, therenormalized expanded image is blended on a pixel-by-pixel basis withthe pre-filtered image. The amount of original blending with thefiltered image can be varied and the blending is performed based uponrelationship:I(x,y,z)=blend*(I(x,y,z)−A(x,y,z))+A(x,y,z);where I(x,y,z) is the renormalized filtered and expanded image, “blend”is a blending parameter, A(x,y,z) is the pre-filtered image.

At step 68, the sharpening is referred to as differential becausedifferent amounts of sharpening are performed on structural pixelsversus non-structural pixels, by reference again to the structure maskobtained at step 60. Contrast or intensity matching is performed,basedupon the original image data. That is, the sharpened image and theoriginal image are both filtered using a low pass (e.g., boxcar) filterand the intensity matching is brought about using the relationship:I(x,y,z)=I(x,y,z)*[lowpassfiltered(I(x,y,z))/lowpassfiltered(A(x,y,z))];where the images I(x,y,z) and A(x,y,z) as are described above.

As noted above, following the foregoing processing, the resulting imagedata may be displayed, stored, and transmitted for viewing. Moreover,more than one image in the image volume may be similarly enhanced, and acollection of images stored for viewing and analysis. The foregoingtechniques provide improved processing in all such contexts,particularly where a size or sampling rate of pixels or voxels issubstantially greater in one dimension than in others.

While only certain features of the invention have been illustrated anddescribed herein, many modifications and changes will occur to thoseskilled in the art. It is, therefore, to be understood that the appendedclaims are intended to cover all such modifications and changes as fallwithin the true spirit of the invention.

1. A method for enhancing a discrete pixel image comprising: using aprocessor to perform the steps of: identifying structural andnon-structural pixels in image data by analysis of the image data in twoand three dimensions; and further processing the image data in twodimensions based upon the identification of structural andnon-structural pixels, wherein the structural pixels are identified byreference to gradient magnitude values computed from three-dimensionaldata and gradient direction values computed from two-dimensional data.2. The method of claim 1, wherein the structural pixels are identifiedbased upon a combination of criteria including three-dimensionalgradient magnitude values and two dimensional gradient direction values.3. The method of claim 2, wherein the structural pixels are identifiedas pixels having a gradient magnitude above a desired threshold and agradient angle above a desired angle.
 4. The method of claim 1, whereinidentifying structural and non-structural pixels is based upon analysispixels neighboring each pixel in three dimensions.
 5. The method ofclaim 1, wherein a structure mask is generated for pixels identified asstructural pixels.
 6. The method of claim 1, wherein the image dataencodes pixels having a first pitch in two dimensions of an image planeand a third pitch greater than the first pitch in a third dimensionorthogonal to the image plane.
 7. The method of claim 1, wherein thefurther processing includes filtering of the image data in twodimensions by reference to the identified structural and non-structuralpixels.
 8. The method of claim 7, wherein the filtering includesorientation smoothing, homogenization smoothing and orientationsharpening.
 9. The method of claim 1, wherein the further processingincludes blending from original image data into processed image data.10. The method of claim 1, wherein structural pixels are distinguishedfrom non-structural pixels based upon three-dimensionally smoothedoriginal image data.
 11. A method for enhancing a discrete pixel imagecomprising: using a processor to perform the steps of: identifyingstructural and non-structural pixels in image data by reference togradient magnitude values computed from three-dimensional data andgradient direction values computed from two dimensional data; andfurther processing the image data in two dimensions based upon theidentification of structural and non-structural pixels.
 12. The methodof claim 11, wherein the gradient magnitude and direction values arecomputed based upon three-dimensionally smoothed image data.
 13. Themethod of claim 11, wherein the structural pixels are identified basedupon a combination of criteria including three-dimensional gradientmagnitude values and two dimensional gradient direction values.
 14. Themethod of claim 13, wherein the structural pixels are identified aspixels having a gradient magnitude above a desired threshold and agradient angle above a desired angle.
 15. A system for enhancing adiscrete pixel image comprising: a memory circuit for storing discretepixel image data; and a processing circuit for identifying structuraland non-structural pixels in image data by analysis of the image data intwo and three dimensions, and for further processing the image data intwo dimensions based upon the identification of structural andnon-structural pixels, wherein the structural pixels are identified byreference to gradient magnitude values computed from three-dimensionaldata and gradient direction values computed from two-dimensional data.16. The system of claim 15, further comprising an image data acquisitionsystem for generating the discrete pixel image data of a subject ofinterest.
 17. The system of claim 16, wherein the image data acquisitionsystem is a computed tomography imaging system.
 18. The system of claim15, wherein the memory circuit is remote from the processing circuit.19. A device comprising: at least one computer-readable medium; andcomputer readable code stored on the at least one computer-readablemedium including instructions for identifying structural andnon-structural pixels in image data by analysis of the image data in twoand three dimensions, and for further processing the image data in twodimensions based upon the identification of structural andnon-structural pixels, wherein the structural pixels are identified byreference to gradient magnitude values computed from three-dimensionaldata and gradient direction values computed from two-dimensional data.20. A device comprising: at least one computer-readable medium; andcomputer readable code stored on the at least one computer-readablemedium including instructions for identifying structural andnon-structural pixels in image data by reference to gradient magnitudevalues computed from three-dimensional data and gradient directionvalues computed from two dimensional data, and for further processingthe image data in two dimensions based upon the identification ofstructural and non-structural pixels.